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2000-319718 

[Name of the Document] Specification 

[Title of the Invention] Image Pickup Apparatus, 

Signal Processing Method 
Thereof, And Storage Medium 
Having Module For Executing 
Signal Processing Thereof 

[What is Claimed is] 
[Claim 1] 

An image pickup apparatus comprising: 
storage means for converting each image pickup data 
obtained by image pickup means into a file to be stored 
in a storage medium, and for storing control data in an 
area different from an area of each file in which said image 
pickup data is stored. 
[Claim 2] 

An image pickup apparatus according to Claim 1, 
further comprising : 

monitor means for reproducing and displaying said 
image pickup data which has been converted into a file and 
stored; 

selection means for selecting a desired image pickup 
data by using said monitor means; 

read-out means for reading out, when the image pickup 
data is selected by said selection means, control data which 
is stored in the same file as that storing the selected 
image pickup data; and 



reproduction means for reproducing said read-out 
image pickup data on the basis of said read-out data. 
[Claim 3] 

An image pickup apparatus according to Claim 1, 
wherein said control data includes white balance control 
data. 

[Claim 4] 

An image pickup apparatus according to Claim 1, 
wherein said control data includes color balance. control 
data . 

[Claim 5] 

An image pickup apparatus according to Claim 1, 
wherein said control data is a representative value of each 
color component data of all or a part of the image pickup 
data obtained by said image pickup means or an average value 
of said each color component. 

[Claim 6] 

An image pickup apparatus according to Claim 5, 
further comprising operation means for calculating, from 
a ratio of each color component of said read-out control 
data, a reciprocal number thereof. 
. [Claim 7] 

An image pickup apparatus according to Claim 3, 
further comprising white balance adjustment means for 
performing white balance adjustment of the image pickup 
data read-out from said storage medium on the basis of said 
white balance control data. 



[Claim 8] 

An image pickup apparatus according to Claim 1, 
wherein said control data is stored in an attached area 
in each file. 

[Claim 9] 

An image pickup apparatus according to Claim 1, 
wherein the image pickup data obtained by the image pickup 
means is matrix-converted into a luminance signal and two 
cooler difference signals after the white balance adjustment 
is performed, to be encoded into a file with predetermined 
specification and stored in a storage medium, and the 
representative value of each color component data or the 
average value data of each color component is stored in 
an attached area in the same file as that storing said image 
pickup data. 

[Claim 10] 

An image pickup apparatus according to any one of 
Claims 1, 3 and 4, wherein said control data includes data 
specific to an image pickup element included in said image 
pickup means . 

[Claim 11] 

An image pickup apparatus according to Claim 1, 
wherein said storage means is detachably attached to said 
image pickup apparatus. 

[Claim 12] 

An image pickup apparatus comprising: 

image pickup means provided with a plurality of 



color filters having different spectroscopic 
characteristics ; 

white balance adjustment means for performing white 
balance adjustment of the image pickup data obtained by 
said image pickup means; 

color balance adjustment means for performing color 
balance adjustment of the image pickup data adjusted by 
said white balance adjustment; 

recording means for converting each image pickup 
data adjusted by said color balance adjustment into a file 
to be stored in a recording medium, and for recording a 
representative value of each color component data or an 
average value data of each color component for all or a 
part of the image pickup data which is originally obtained 
by said image pickup means in an area of each file different 
from an area for storing said image pickup data adjusted 
by the color balance adjustment; 

monitor means for reproducing and displaying said 
image pickup data which has been converted into a file and 
stored; 

first selection means for selecting desired image 
pickup data by using said monitor means; 

read-out means for reading out, when the image pickup 
data is selected by said first selection means, the data 
of said representative value or said average value which 
is stored in the same file as that storing this image pickup 
data; 
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first operation means for calculating a control 
value for said white balance adjustment means on the basis 
of said read-out data; 

second operation means for calculating a control 
value for said color balance adjustment means on the basis 
of said read-out data; 

second selection means for selecting whether said 
read-out data is used as a control value for said white 
balance adjustment means or a control value for said color 
balance adjustment means; and 

setting means for setting a control value which 
is selected by said second selection means and calculated, 

wherein the white balance adjustment or the color 
balance adjustment is performed by the use of the control 
value set by said setting means. 

[Claim 13] 

An image pickup apparatus according to Claim 12, 
wherein the first operation means calculates a ratio of 
each color component of said read-out data, and a reciprocal 
number thereof. 

[Claim 14] 

An image pickup apparatus according to Claim 12 
or 13, wherein the white balance adjustment means performs 
white balance adjustment by multiplying the set control 
value by each color component of the image pickup data obtained 
by the image pickup means. 

[Claim 15] 
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An image pickup apparatus according to any one of 
Claims 12 to 14 , wherein the second operation means calculates 
a ratio of each color component of the read-out data. 

[Claim 16] 

An image pickup apparatus according to any one of 
Claims 12 to 15, wherein the color balance adjustment means 
performs color balance adjustment by mult iplying the control 
value by each color component of the image pickup data ad j us ted 
by the white balance adjustment. 

[Claim 17] 

An image pickup apparatus comprising: 

image pickup means provided with a plurality of 
color filters having different spectroscopic 
characteristics ; 

first adjustment means for performing white balance 
adjustment or color balance adjustment of the image pickup 
data obtained by said image pickup means; 

a memory for storing image pickup data for each 
color component data with respect to specific plural light 
sources picked up by a referential image pickup apparatus 
and image pickup data for each color component data with 
respect to said specific plural light sources intrinsic 
to the image pickup apparatus; 

recording means for converting each image pickup 
data adjusted by said white balance adjustment or said color 
balance adjustment into a file to be stored in a recording 
medium, and for recording a representative value of each 
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color component data or an average value data of each color 
component for all or a part of the image pickup data which 
is originally obtained by said image pickup means, image 
pickup data for each color component data with respect to 
specific plural light sources picked up by a referential 
image pickup apparatus , and image pickup data for each color 
component data with respect to said specific plural light 
sources intrinsic to the image pickup apparatus stored in 
said memory, in an area of each file different from an area 
for storing said image pickup data adjusted by the white 
balance adjustment or the color balance adjustment; 

monitor means for reproducing and displaying said 
image pickup data which has been converted into a file and 
stored; 

selection means for selecting desired image pickup 
data by using said monitor means; 

first read-out means for reading out, when image 
pickup data is selected by said selection means, said 
representative value or said average value data which is 
stored in the same file as that storing this image pickup 
data, and the image pickup data for each color component 
data with respect to specific plural light sources picked 
up by a referential image pickup apparatus and the image 
pickup data for each color component data with respect to 
said specific plural light sources intrinsic to the image 
pickup apparatus which are stored in the same file; 

second read-out means for reading out the image 
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pickup data for each color component data with respect to 
specific plural light sources picked up by a referential 
image pickup apparatus and the image pickup data for each 
color component data with respect to said specific plural 
light sources intrinsic to the image pickup apparatus which 
are stored in said memory; 

second adjustment means for adjusting the 
representative value or the average value data read-out 
by said first read-out means on the basis of the image pickup 
data for each color component data with respect to the specific 
plural light sources picked up by a referential image pickup 
apparatus, the image pickup data for each color component 
data with respect to said specific plural light sources 
intrinsic to the image pickup apparatus which are read-out 
by said first read-out means, the image pickup data for 
each color component data with respect to the specific plural 
light sources picked up by a referential image pickup 
apparatus, and the image pickup data for each color component 
data with respect to said specific plural light sources 
intrinsic to the image pickup apparatus which are read-out 
by said second read-out means; 

operation means for calculating a control value 
for said first adjustment means on the basis of said 
representative value or said average value data adjusted 
by said second adjustment means; and 

setting means for setting the control value which 
is operated by said operation means, 
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wherein said first adjustment means performs the 
white balance adjustment or the color balance adjustment 
by the use of the control value set by said setting means. 

[Claim 18] 

An image pickup apparatus according to Claim 17 , 
wherein the operation means calculates a ratio of each color 
component of the adjusted data of the representative value 
or the average data or a reciprocal number of the ratio. 

[Claim 19] 

An image pickup apparatus according to Claim 17 
or 18, wherein the first adjustments means performs the 
white balance adjustment or the color balance adjustment 
by multiplying the set control value by each color component 
of the image pickup data obtained by the image pickup means. 

[Claim 20] 

A signal processing method of an image pickup 
apparatus comprising : 

a storing step of converting each image pickup data 
obtained in an image pickup step into a file to be stored 
in a storage medium and, storing a control data in an area 
of the file different from an area storing said image pickup 
data . 

[Claim 21] 

A signal processing method according to Claim 20, 
wherein said control data includes white balance control 
data. 

[Claim 22] 
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An signal processing method according to Claim 20, 
wherein said control data includes color balance control 
data . 

[Claim 23] 

A signal processing method according to Claim 20, 
wherein said control data is a representative value of each 
color component data with respect to all or a part of the 
image pickup data obtained by said image pickup means or 
an average value of said each color component. 

[Claim 24] 

A signal processing method according to Claim 21, 
further comprising white balance adjustment means for 
performing white balance adjustment of the image pickup 
data read-out from said storage medium on the basis of said 
white balance control data. 

[Claim 25] 

A signal processing method according according 
to Claim 20, wherein said control data is stored in an attached 
area in each file. 

[Claim 26] 

A signal processing method according according 
to Claim 20, wherein the image pickup data obtained in the 
image pickup step is matrix-converted into a luminance signal 
and two cooler difference signals after the white balance 
adjustment is performed to be encoded into a file with 
predetermined specification and stored in a storage medium, 
and the representative value of each color component data 
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or the average value data for each color component is stored 
in an attached area in the same file as that storing said 
image pickup data. 

[Claim 27] 

A signal processing method according to any one 
of Claims 20, 22 and 23, wherein said control data includes 
data specific to an image pickup element included in said 
image pickup means. 

[Claim 28] 

A signal processing method according toClaim20, 
wherein said storage means is detachably attached to said 
image pickup apparatus. 

[Claim 29] 

A signal processing method of an image pickup 
apparatus comprising the steps of: 

performing white balance adjustment of image pickup 
data which is obtained by image pickup means provided with 
a plurality of color filters having different spectroscopic 
characteristics by the use of white balance adj ustment means ; 

performing color balance adjustment of the image 
pickup data adjusted by said white balance adjustment by 
the use of color balance adjustment means; 

converting each image pickup data adjusted by said 
color balance adjustment into a file to be stored in a 
recording medium and recording a representative value of 
each color component data or an average value data of each 
color component for all or a part of the image pickup data 



- 12 - 



which is originally obtained by said image pickup means 
in an area of each file different from an area for recording 
said image pickup data adjusted by the color balance 
adjustment; 

reading out, when desired image pickup data is 
selected by the use of monitor means for reproducing and 
displaying said image pickup data which has been converted 
into a file and stored, said representative value or said 
average value data which is stored in the same file as that 
storing this image pickup data; 

calculating a control value for said white balance 
adjustment means or a control value for said color balance 
adjustment means on the basis of said read-out data; 

setting a control value which is calculated by 
selecting whether said read-out data is used as a control 
value for said white balance adjustment means or as a control 
value for said color balance adjustment means; and 

performing the white balance adjustment or the color 
balance adjustment by the use of said set control value. 

[Claim 30] 

A signal processing method of an image pickup 
apparatus comprising the steps of: 

performing white balance adj ustment or color balance 
adjustment of the image pickup data obtained by image pickup 
means provided with a plurality of color filters having 
different spectroscopic characteristics by the use of first 
adjustment means; 
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storing image pickup data for each color component 
data with respect to specific plural light sources picked 
up by a referential image pickup apparatus and image pickup 
data for each color component data with respect to said 
specific plural light sources intrinsic to the image pickup 
apparatus in a memory; 

converting each image pickup data adjusted by said 
white balance adjustment or said color balance adjustment 
into a file to be stored in a recording medium and recording 
a representative value of each color component data or an 
average value data of each color component for all or a 
part of the image pickup data which is originally obtained 
by said image pickup means, the image pickup data for each 
color component data with respect to specific plural light 
sources picked up by a referential image pickup apparatus, 
and the image pickup data for each color component data 
with respect to said specific plural light sources intrinsic 
to the image pickup apparatus recorded in said memory in 
an area of each file different from an area storing said 
image pickup data adjusted by the white balance adjustment 
or the color balance adjustment; 

reading out, when desired image pickup data is 
selected by the use of monitor means for reproducing and 
displaying said image pickup data which has been converted 
into a file and stored, said representative value or said 
average value data which is stored in the same file as that 
storing this image pickup data, and the image pickup data 
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for each color component data with respect to specific plural 
light sources picked up by a referential image pickup 
apparatus and the image pickup data for each color component 
data with respect to said specific plural light sources 
intrinsic to the image pickup apparatus which are recorded 
in the same file by the use of first read-out means; 

reading out the image pickup data for each color 
component data with respect to specific plural light sources 
picked up by a referential image pickup apparatus and the 
image pickup data for each color component data with respect 
to said specific plural light sources intrinsic to the image 
pickup apparatus which are stored in said memory by the 
use of second read-out means; 

adjusting said representative value or said average 
value data read out by said first read-out means on the 
basis of the image pickup data for each color component 
data with respect to specific plural light sources picked 
up by a referential image pickup apparatus, the image pickup 
data for each color component data with respect to said 
specific plural light sources intrinsic to the image pickup 
apparatus which are read out by said first read-out means, 
the image pickup data for each color component data with 
respect to specific plural light sources picked up by a 
referential image pickup apparatus, and the image pickup 
data for each color component data with respect to said 
specific plural light sources intrinsic to the image pickup 
apparatus which are read out by said second read-out means 
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by the use of second adjustment means; 

calculating a control value for said first adjustment 
means on the basis of said representative value or said 
average value data adjusted by said second adjustment means; 

setting said calculated control value; and 

performing the white balance adjustment or the color 
balance adjustment by the use of the control value set by 
said first adjustment means. 

[Claim 31] 

A storage medium for storing a module comprising: 
a storing step for converting each image pickup 
data obtained at the image pickup step into a file to be 
stored in the storage medium and for storing a control data 
in an area of each file different from an area storing said 
image pickup data. 

[Claim 32] 

A storage medium according to Claim 31, wherein 
said control data includes white balance control data. 
[Claim 33] 

A storage medium according to Claim 31, wherein 
said control data includes color balance control data. 
[Claim 34] 

A storage medium according to Claim 31, wherein 
said control data is a representative value of each color 
component data with respect to all or a part of the image 
pickup data obtained by said image pickup means or an average 
value of said each color component. 
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[Claim 35] 

A storage medium according to Claim 32, further 
comprising white balance adjustment means for performing 
white balance adjustment of the image pickup data read out 
from said storage medium. 

[Claim 36] 

A storage medium according to Claim 31, wherein 
said control data is stored in an attached area in each 
file. 

[Claim 37] 

A storage medium according to Claim 31, wherein 
the image pickup data obtained at said image pickup step 
is matrix-converted into a luminance signal and two cooler 
difference signals after the white balance adjustment is 
performed to be encoded into a file with predetermined 
specification and stored in a storage medium, and the 
representative value of each color component data or the 
average value data of each color component is stored in 
an attached area in the same file as that storing said image 
pickup data. 

[Claim 38] 

A storage medium according to any one of Claims 
31, 33 and 34, wherein said control data includes data specif ic 
to an image pickup element included in said image pickup 
means . 

[Claim 39] 

A storage medium according to Claim 31, wherein 
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said storage means is detachably attached to said image 
pickup apparatus. 

[Detailed Description of the Invention] 
[0001] 

[Field of the Industrial Utilization] 

The present invention relates to an image pickup 
apparatus for adjusting tone of image data picked up by 
image pickup means having a plurality of color filters, 
a signal processing method thereof and a storage medium 
having a module for executing the signal processing. 
[0002] 
[Prior Art] 

To output image data picked up by an image pickup 
element having a plurality of color filters, the picked-up 
image data undergoes white balance adjustment, and the color 
temperature of a light source in the picked-up image so 
that the image is adjusted to be achromatic, thereby 
performing faithful color reproduction of the picked-up 
image . 
[0003] 

The white balance ad j us tment methods include a method 
of determining the color temperature of a light source using 
a calorimetric element and a method of searching for an 
achromatic portion in the picked-up image and determining 
the color temperature of the achromatic portion. These 
methods, however, cannot accurately adj ust the white balance 
due to detection errors of colors of the light sources- 
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[0004] 

As one of the most accurate white balance adjustment 
methods, a white balance adjustment method is available 
in which an achromatic object is picked up (this operation 
is called white sheet image pickup) , a control value is 
obtained so as to equalize the magnitudes of color components 
of the picked-up image, and white balance adjustment is 
performed for the subsequent image pickup using the control 
value . This white balance adj ustment method will be referred 
to as a manual white balance (MWB) hereinafter. 
[0005] 

A conventional MWB control method using white sheet 
image pickup is shown in Fig. 11. Fig. 11 shows the 
conventional white balance adjustment method of partially 
extracting data (to be referred to white sheet data 
hereinafter) from an achromatic image (to be referred to 
as a white sheet image hereinafter) picked up by image pickup 
means having a plurality of color filters, recording the 
extracted white sheet data on a recording medium, reading 
the white sheet data from the medium in the subsequent image 
pickup to calculate a white balance control value, and 
adjusting the white balance using the calculated control 
value . 
[0006] 

Each of the plurality of color filters of the image 
pickup element is made up of primary colors of R, Gl, G2, 
and B, as shown in Fig. 5. Image pickup data input from 
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an image pickup data input terminal 1001 is processed by 
a main image processing unit 1002, encoding processing unit 
1003, and image filing unit 1004. The processed data is 
recorded on a recording medium as image data by a 
medium-recording unit 1005. 
[0007] 

Independently of this normal image recording process , 
part of the image pickup data is extracted by an image data 
extraction unit as MWB white sheet data. The extracted 
MWB white sheet data is recorded in the MWB data area of 
the recording medium. 
[0008] 

When the white balance mode is set in the MWB in 
the subsequent image pickup, a medium-reproduction unit 
1007 in the image pickup apparatus reproduces the white 
data recorded on the medium. A white balance control value 
read-out unit 1008 reads out the white data. A white balance 
control value operating unit 1009 calculates the R, Gl, 
G2, and B white balance control values so as to equalize 
the magnitudes of the average values of the R, Gl, G2 , and 
B color signals of the white data. 
[0009] 

The white balance control values obtained by the 
white balance control value operating unit 1009 are set 
for white balance adjustment of the image pickup apparatus 
by a white balance control value setting unit 1010. Using 
the set control values, a white balance adjustment unit 
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1014 adjusts the white balance of the image pickup data 
input from an image pickup data input terminal 1013 in the 
subsequent image pickup. A color processing unit 1015 
performs final color adjustment such as conversion of a 
color matrix or color correction processing. An encoding 
processing unit 1016 encodes the color-adjusted data into 
a recording format. The data output from the encoding 
processing unit 1016 is converted into a recording file 
by an image filing unit 1017 . The resultant file is recorded 
on a- recording medium 1018 as a pickup image file. 
[0010] 

As shown in Fig. 12, in the MWB mode, a white balance 
control value operating unit 1102 directly calculates white 
balance control values from the white sheet data picked 
up by an image pickup apparatus (input from an image pickup 
data input terminal 1101) . The calculated control values 
are set in the image pickup apparatus by a white balance 
control value setting unit 1103 and used as the control 
values in subsequent image pickup for a white balance 
adjustment unit 1114. 
[0011] 

By using the set control values, the white balance 
adjustment unit 1114 adjusts the white balance of the image 
pickup data input from an image pickup data input terminal 
1113 in the subsequent image pickup. A color processing 
unit 1115 performs final color adjustment such as color 
matrix conversion and color correction processing. An 
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output from the color processing unit 1115 is encoded into 
a recording format by an encoding processing unit 1116. 

An output from the encoding processing unit 1116 is converted 
into a recording file by an image filing unit 1117. The 
resultant file is recorded on a recording medium 1118 as 
a pickup image file. 

[0012] 

[Problems to be Solved by the Invention] 

In the prior arts described above, however, white 
sheet data is written in a specific area called the MWB 
data area of a recording medium. The recording medium is 
inserted into an image pickup apparatus to read out the 
white data in the MWB mode. It is difficult to store a 
plurality of types of MWB white sheet data on one recording 
medium, manage them, and read out desired white data. The 
white data cannot be conveniently used. 

[0013] 

The method of directly calculating the white balance 
control values upon picking up image pickup data and setting 
them in the image pickup apparatus cannot set an another 
MWB mode when the white data is prepared once and continuously 
used, resulting in inconvenience. The white data cannot 
be conveniently used either. 
[0014] 

When white sheet data picked up by a given image 
pickup apparatus is used as white data for another image 
pickup apparatus in the above system, accurate white balance 
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adjustment cannot be performed due to the influence of 
variations in spectral characteristics of the image pickup 
elements. The white sheet data cannot be conveniently used 
either . 
[0015] 

The present invention has been made in consideration 
of the conventional problems described above, and has its 
object to provide an image pickup apparatus in which white 
sheet data used for color correction of a picked-up image 
can be conveniently used, a signal processing method for 
the image pickup apparatus, and a recording medium having 
a module for executing the signal processing of the signal 
processing method . 
[0016] 

[Means for Solving the Problems] 

An image pickup apparatus , a signal processing method 
thereof, and a storage medium having a module for executing 
a signal processing thereof according to the present 
invention are arranged as follows. 

[0017] 

(1) There is provided an image pickup apparatus 
comprising storage means for converting each image pickup 
data obtained by image pickup means into a file to be stored 
in a storage medium, and for storing control data in an 
area of each file different from an area in which the image 
pickup data is stored. 
[0018] 



- 23 - 



(2) The image pickup apparatus set forth in the 
above (1) further comprises: monitor means for reproducing 
and displaying the image pickup data which has been converted 
into a file and stored; selection means for selecting a 
desired image pickup data by using the monitor means ; read-out 
means for reading out, when the image pickup data is selected 
by the selection means, control data which is stored in 
the same file as that storing the selected image pickup 
data; and reproduction means for reproducing the read-out 
image pickup data on the basis of the read-out data. 
[0019] 

(3) In the image pickup apparatus set forth in 
the above ( 1 ) , the control data includes white balance control 
data . 

[0020] 

(4) In the image pickup apparatus set forth in 
the above (1) , the control data includes color balance control 
data . 

[0021] 

(5) In the image pickup apparatus set forth in 
the above (1), the control data is a representative value 
of each color component data of all or a part of the image 
pickup data obtained by the image pickup means or an average 
value of the each color component. 

[0022] 

(6) The image pickup apparatus set forth in the 
above (5) further comprises operation means for calculating, 
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from a ratio of each color component of the read-out control 
data, a reciprocal number thereof. 
[0023] 

(7) The image pickup apparatus set forth in the 
above (3) further comprises white balance adjustment means 
for performing white balance adjustment of the image pickup 
data read-out from the storage medium on the basis of the 
white balance control data. 

[0024] 

(8) In the image pickup apparatus set forth in 
the above (1), the control data is stored in an attached 
area in each file. 

[0025] 

(9) In the image pickup apparatus set forth in 
the above (1), the image pickup data obtained by the image 
pickup means is matrix-converted into a luminance signal 
and two cooler difference signals after the white balance 
adjustment is performed, to be encoded into a file with 
predetermined specification and stored in a storage medium, 
and the representative value of each color component data 
or the average value data of each color component is stored 
in an attached area in the same file as that storing the 
image pickup data. 

[0026] 

(10) In the image pickup apparatus set forth in 
any one of the above (1), (3) and (4), the control data 
includes data specific to an image pickup element included 
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in the image pickup means. 
[0027] 

(11) In the image pickup apparatus set forth in 
the above (1) , the storage means is detachably attached 
to the image pickup apparatus. 

[0028] 

(12) There is provided an image pickup apparatus 
comprising: image pickup means provided with a plurality 
of color filters having different spectroscopic 
characteristics; white balance adjustment means for 
performing white balance adjustment of the image pickup 
data obtained by the image pickup means; color balance 
adjustment means for performing color balance adjustment 
of the image pickup data adjusted by the white balance 
adjustment ; recording means for converting each image pickup 
data adjusted by the color balance adjustment into a file 
to be stored in a recording medium, and for recording a 
representative value of each color component data or an 
average value data of each color component for all or a 
part of the image pickup data which is originally obtained 
by the image pickup means in an area of each file different 
from an area for storing the image pickup data adjusted 
by the color balance adjustment; monitor means for 
reproducing and displaying the image pickup data which has 
been converted into a file and stored; first selection means 
for selecting desired image pickup data by using the monitor 
means; read-out means for reading out, when the image pickup 
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data is selected by the first selection means, the data 
of the representative value or the average value which is 
stored in the same file as that storing this image pickup 
data; first operation means for calculating a control value 
for the white balance adjustment means on the basis of the 
read-out data; second operation means for calculating a 
control value for the color balance adjustment means on 
the basis of the read-out data; second selection means for 
selecting whether the read-out data is used as a control 
value for the white balance adjustment means or a control 
value for the color balance adjustment means; and setting 
means for setting a control value which is selected by the 
second selection means and calculated, so that the white 
balance adjustment or the color balance adjustment is 
performed by the use of the control value set by the setting 
means . 
[0029] 

(13) In the image pickup apparatus set forth in 
the above (12) , the first operation means calculates a ratio 
of each color component of the read-out data, and a reciprocal 
number thereof. 

[0030] 

(14) In the image pickup apparatus set forth in 
the above (12) or (13), the white balance adjustment means 
performs white balance adjustment by multiplying the set 
control value by each color component of the image pickup 
data obtained by the image pickup means. 
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[0031] 

(15) In the image pickup apparatus set forth in 
any one of the above (12) to (14), the second operation 
means is to calculates a ratio of each color component of 
the read-out data. 

[0032] 

(16) In the image pickup apparatus set forth in 
any one of the above (12) to (15), the color balance ad j ustment 
means performs color balance adjustment by multiplying the 
control value by each color component of the image pickup 
data adjusted by the white balance adjustment. 

[0033] 

(17) There is provided an image pickup apparatus 
comprising: image pickup means provided with a plurality 
of color filters having different spectroscopic 
characteristics ; first ad j ustment means for performing white 
balance adjustment or color balance adjustment of the image 
pickup data obtained by the image pickup means; a memory 
for storing image pickup data for each color component data 
with respect to specific plural light sources picked up 
by a referential image pickup apparatus and image pickup 
data for each color component data with respect to the specific 
plural light sources intrinsic to the image pickup apparatus ; 
recording means for converting" each image pickup data 
adjusted by the white balance adjustment or the color balance 
adjustment into a file to be stored in a recording medium, 
and for recording a representative value of each color 
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component data or an average value data of each color component 
for all or a part of the image pickup data which is originally 
obtained by the image pickup means, image pickup data for 
each color component data with respect to specific plural 
light sources picked up by a referential image pickup 
apparatus, and image pickup data for each color component 
data with respect to the specific plural light sources 
intrinsic to the image pickup apparatus stored in the memory, 
in an area of each file different from an area for storing 
the image pickup data adj us ted by the white balance ad j ustment 
or the color balance adjustment; monitor means for 
reproducing and displaying the image pickup data which has 
been converted into a file and stored; selection means for 
selecting desired image pickup data by using the monitor 
means; first read-out means for reading out, when image 
pickup data is selected by the selection means, the 
representative value or the average value data which is 
stored in the same file as that storing this image pickup 
data, and the image pickup data for each color component 
data with respect to specific plural light sources picked 
up by a referential image pickup apparatus and the image 
pickup data for each color component data with respect to 
the specific plural light sources intrinsic to the image 
pickup apparatus which are stored in the same file; second 
read-out means for reading out the image pickup data for 
each color component data with respect to specific plural 
light sources picked up by a referential image pickup 
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apparatus and the image pickup data for each color component 
data with respect to the specific plural light sources 
intrinsic to the image pickup apparatus which are stored 
in the memory; second adjustment means for adjusting the 
representative value or the average value data read-out 
by the first read-out means on the basis of the image pickup 
data for each color component data with respect to the specific 
plural light sources picked up by a referential image pickup 
apparatus, the image pickup data for each color component 
data with respect to the specific plural light sources 
intrinsic to the image pickup apparatus which are read-out 
by the first read-out means, the image pickup data for each 
color component data with respect to the specific plural 
light sources picked up by a referential image pickup 
apparatus, and the image pickup data for each color component 
data with respect to the specific plural light sources 
intrinsic to the image pickup apparatus which are read-out 
by the second read-out means ; operation means for calculating 
a control value for the first adjustment means on the basis 
of the representative value or the average value data adjusted 
by the second adjustment means ; and setting means for setting 
the control value which is operated by the operation means, 
so that the first adjustment means performs the white balance 
adjustment or the color balance adjustment by the use of 
the control value set by the setting means. 
[0034] 

(18) In the image pickup apparatus set forth in 
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the above (17), the operation means calculates a ratio of 
each color component of the adjusted data of the 
representative value or the average data or a reciprocal 
number of the ratio. 
[0035] 

(19) In the image pickup apparatus set forth in 
the above (17) or (18) , the first adjustments means performs 
the white balance adjustment or the color balance adjustment 
by multiplying the set control value by each color component 
of the image pickup data obtained by the image pickup means. 
[0036] 

(20) There is provided a signal processing method 
of an image pickup apparatus comprising: a storing step 
of converting each image pickup data obtained in an image 
pickup step into a file to be stored in a storage medium 
and, of storing a control data in an area different from 
an area storing the image pickup data of the file. 
[0037] 

(21) In the signal processing method set forth 
in the above (20) , the control includes white balance control 
data . 

[0038] 

(22) In the signal processing method set forth 
in the above (20), the control data includes color balance 
control data. 

[0039] 

(23) In the signal processing method set forth 
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in the above (20), the control data is a representative 
value of each color component data with respect to all or 
a part of the image pickup data obtained by the image pickup 
means or an average value of the each color component. 
[0040] 

(24) The signal processing method set forth in 
the above (21) further comprises white balance adjustment 
means for performing white balance adjustment of the image 
pickup data read-out from the storage medium on the basis 
of the white balance control data. 

[0041] 

(25) In the signal processing method set forth 
in the above (20) , the control data is stored in an attached 
area in each file. 

[0042] 

(26) In the signal processing method set forth 
in the above (20), the image pickup data obtained in the 
image pickup step is matrix-converted into a luminance signal 
and two cooler difference signals after the white balance 
adjustment is performed to be encoded into a file with 
predetermined specification and stored in a storage medium, 
and the representative value of each color component data 
or the average value data for each color component is stored 
in an attached area in the same file as that storing the 
image pickup data. 

[0043] 

(27) In the signal processing method set forth 
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in any one of the above (20), (22) and (23), the control 
data includes data specific to an image pickup element 
included in the image pickup means. 

(28) In the signal processing method set forth 
in the above (20) , the storage means is detachably attached 
to the image pickup apparatus. 

[0044] 

(29) There is provided a signal processing method 
of an image pickup apparatus comprising the steps of: 
performing white balance adjustment of image pickup data 
which is obtained by image pickup means provided with a 
plurality of color filters having different spectroscopic 
characteristics by the use of white balance adjustment means ; 
performing color balance adjustment of the image pickup 
data adjusted by the white balance adjustment by the use 
of color balance adjustment means; converting each image 
pickup data adjusted by the color balance adjustment into 
a file to be stored in a recording medium and recording 
a representative value of each color component data or an 
average value data of each color component for all or a 
part of the image pickup data which is originally obtained 
by the image pickup means in an area of each file different 
from an area for recording the image pickup data adjusted 
by the color balance adjustment; reading out, when desired 
image pickup data is selected by the use of monitor means 
for reproducing and displaying the image pickup data which 
has been converted into a file and stored, the representative 
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value or the average value data which is stored in the same 
file as that storing this image pickup data; calculating 
a control value for the white balance adjustment means or 
a control value for the color balance adjustment means on 
the basis of the read-out data; setting a control value 
which is calculated by selecting whether the read-out data 
is used as a control value for the white balance adjustment 
means or as a control value for the color balance adjustment 
means; and performing the white balance adjustment or the 
color balance adjustment by the use of the set control value. 
[0045] 

(30) There is provided a signal processing method 
of an image pickup apparatus comprising the steps of: 
performing white balance adjustment or color balance 
adjustment of the image pickup data obtained by image pickup 
means provided with a plurality of color filters having 
different spectroscopic characteristics by the use of first 
adjustment means; storing image pickup data for each color 
component data with respect to specific plural light sources 
picked up by a referential image pickup apparatus and image 
pickup data for each color component data with respect to 
the specific plural light sources intrinsic to the image 
pickup apparatus in a memory; converting each image pickup 
data adjusted by the white balance adjustment or the color 
balance adjustment into a' file to be stored in a recording 
medium and recording a representative value of each color 
component data or an average value data of each color component 
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for all or a part of the image pickup data which is originally 
obtained by the image pickup means, the image pickup data 
for each color component data with respect to specific plural 
light sources picked up by a referential image pickup 
apparatus, and the image pickup data for each color component 
data with respect to the specific plural light sources 
intrinsic to the image pickup apparatus recorded in the 
memory in an area of each file different from an area storing 
the image pickup data adj us ted by the white balance ad j ustment 
or the color balance adjustment; reading out, when desired 
image pickup data is selected by the use of monitor means 
for reproducing and displaying the image pickup data which 
has been converted into a file and stored, the representative 
value or the average value data which is stored in the same 
file as that storing this image pickup data, and the image 
pickup data for each color component data with respect to 
specific plural light sources picked up by a referential 
image pickup apparatus and the image pickup data for each 
color component data with respect to the specific plural 
light sources intrinsic to the image pickup apparatus which 
are recorded in the same file by the use of first read-out 
means; reading out the image pickup data for each color 
component data with respect to specific plural light sources 
picked up by a referential image pickup apparatus and the 
image pickup data for each color component data with respect 
to the specific plural light sources intrinsic to the image 
pickup apparatus which are stored in the memory by the use 
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of second read-out means; adjusting the representative value 
or the average value data read out by the first read-out 
means on the basis of the image pickup data for each color 
component data with respect to specific plural light sources 
picked up by a referential image pickup apparatus, the image 
pickup data for each color component data with respect to 
the specific plural light sources intrinsic to the image 
pickup apparatus which are read out by the first read-out 
means, the image pickup data for each color component data 
with respect to specific plural light sources picked up 
by a referential image pickup apparatus, and the image pickup 
data for each color component data with respect to the specific 
plural light sources intrinsic to the image pickup apparatus 
which are read out by the second read-out means by the use 
of second adjustment means; calculating a control value 
for the first adjustment means on the basis of the 
representative value or the average value data adjusted 
by the second adjustment means; setting the calculated 
control value; and performing the white balance adjustment 
or the color balance adjustment by the use of the control 
value set by the first adjustment means. 
[0046] 

( 31 ) There is provided a storage medium for storing 
a module comprising: a storing step for converting each 
image pickup data obtained at the image pickup step into 
a file to be stored in the storage medium and for storing 
a control data in an area of each file different from an 
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area storing the image pickup data. 
[0047] 

(32) In the storage medium set forth in the above 
(31) , the control data includes white balance control data. 
[0048] 

(33) In the storage medium set forth in the above 
(31) , the control data includes color balance control data. 
[0049] 

(34) In the storage medium set forth in the above 

(31) , the control data is a representative value of each 
color component data with respect to all or a part of the 
image pickup data obtained by the image pickup means or 
an average value of the each color component. 

[0050] 

(35) The storage medium set forth in the above 

(32) further comprises white balance adjustment means for 
performing white balance adjustment of the image pickup 
data read out from the storage medium. 

[0051] 

(36) In the storage medium set forth in the above 
(31) , the control data is stored in an attached area in 
each file. 

[0052] 

(37) In the storage medium set forth in the above 
(31) , the image pickup data obtained at the image pickup 
step is matrix-converted into a luminance signal and two 
cooler difference signals after the white balance adjustment 
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is performed to be encoded into a file with predetermined 
specification and stored in a storage medium, and the 
representative value of each color component data or the 
average value data of each color component is stored in 
an attached area in the same file as that storing the image 
pickup data. 
[0053] 

(38) In the storage medium set forth in any one 
of the above (31), (33) and (34), the control data includes 
data specific to an image pickup element included in the 
image pickup means. 

[0054] 

(39) In the storage medium set forth in the above 
(31) , the storage means is detachably attached to the image 
pickup apparatus. 

[0055] 

[Detailed Description of the Preferred Embodiments] 

(First Embodiment) 

Fig. 1 is a block diagram showing the main part 
of an image pickup apparatus according to the first embodiment . 

This embodiment will exemplify a method of recording a 
part of image pickup data as white sheet data into an attached 
data area of each image file when each pickup image is recorded 
on a recording medium. 
[0056] 

Referring to Fig. 1, an image pickup data input 
terminal 101 receives image pickup data obtained by an image 
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pickup element having four, R, Gl, G2, and B, color filters 

and converted into digital data. 

[0057] 

This image pickup data is processed in a main image 
processing unit 102 for various image processing operations 
for recording with high image quality as in a digital image 
recording apparatus (image pickup apparatus) such as a 
general digital camera. The processed image pickup data 
is then encoded by an encoding processing unit 103 in 
accordance with a JPEG recording format. 
[0058] 

On the other hand, a part of image pickup data, 
e.g., pixel data of 64 pixels in the central portion of 
the two-dimensional image space of one picture is extracted 
by a white sheet data extraction unit 104, as shown in Fig. 9. 

An averaging processing unit 105 obtains an average value 
of 16-pixel data of each color component signal of R, Gl, 
G2, or B. 
[0059] 

An image filing unit 106 forms an image file together 
with the original encoded image data. The average value 
is allocated to an area attached to the predetermined MWB 
white sheet data of the image file, and is recorded by a 
medium recording unit 107 as one image file together with 
the original encoded image data. 
[0060] 

The above-described operation is inevitably 
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performed at every image picking up operation including 
a general image picking up operation. In the same manner , 
the white sheet image picking up operation is performed 
as one of the general image picking up operation. Since 
the white sheet data is obtained by extracting an image 
of 64 pixels in the central portion of the picture, any 
color image except white such as a white sheet can be picked 
up around the 64-pixel area. 
[0061] 

A technique for setting a white balance control 
value in the MWB mode in the image pickup apparatus will 
be described below. A medium on which an image file is 
recorded by the above recording scheme is loaded in the 
image pickup apparatus, and image file reproduction is done 
using a user interface shown in Fig. 7. 
[0062] 

A PLAY button 703 shown in Fig. 7 is depressed to 
reproduce and display this image data (image pickup data) 
of the image file on a reproduction monitor 701. 
[0063] 

The operator sequentially switches a plurality of 
files with file selection switches 704 and 705 while viewing 
the reproduction monitor 701 and selects one image file 
to be used in MWB. 
[0064] 

When an image file is reproduced and monitored, 
the white sheet data is also read out from an attached white 
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sheet data area by a white sheet data reproduction unit 

(read-out means) 110. 

[0065] 

File selection is done as follows. While a file 
in which white data to be used for MWB is written is being 
displayed on the monitor, a SELECT switch 708 is depressed 
to cause a white sheet data image selection unit 109 to 
select a desired one. 
[0066] 

As indicated by the reproduction monitor 701 in 
Fig. 7, if the image pickup date and place are written in 
a location except a white data recording area 702 when 
intentionally picking up an image used in MWB, operation 
for the white data can be further facilitated. 
[0067] 

In accordance with a signal indicating that the 
file is selected, the white sheet data image selection unit 
109 selects the white sheet data of an image file being 
reproduced. A white balance control value operating unit 
111 calculates a white balance control value on the basis 
of the white data. 
[0068] 

White balance calculations are done as follows. 
The ratio of average values of R, Gl, G2, and B is calculated, 
and the reciprocal of the ratio of R and B using the Gl 
and G2 average values as the median is calculated as a white 
balance control values. 
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[0069] 

A white balance control value setting unit 112 sets 
the resultant white balance control value as the MWB control 
value in the image pickup apparatus. 
[0070] 

When image pickup operation is performed using the 
MWB control value set as described above, a white balance 
adjustment unit 114 multiplies the set MWB control value 
with each color component, input from an image pickup data 
input terminal 113, of an image signal output from the image 
pickup element and converted into digital data, thereby 
adjusting the white balance. 
[0071] 

The white balance-adjusted signal undergoes color 
processing in a color processing unit 115 and encoding 
processing in an encoding processing unit 116. An image 
filing unit 117 converts the encoded image signal into an 
image file together with the white sheet data extracted 
from the image pickup data. A medium-recording unit 118 
records the image file on a recording medium. 
[0072] 

As described above, there is provided a function 
of setting in the image pickup apparatus the MWB control 
value obtained from the white sheet data recorded together 
with the image pickup data in the image file. White data 
recorded in a plurality of scenes can easily be repeatedly 
used. Image pickup operation for setting an MWB control 
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value upon a change in scene need not be done, thereby 

facilitating the use of MWB adjustment. 

[0073] 

As described above, according to the first embodiment, 
a plurality of white sheet data can be managed using one 
medium and accessed easily, thereby improving convenience 
in use of MWB white sheet data. 
[0074] 

( Second Embodiment ) 

Fig. 2 is a block diagram showing the main part 
of an image pickup apparatus according to the second 
embodiment. This embodiment will explain a method of 
recording part of image pickup data as color adjustment 
data (to be referred to as color balance data hereinafter) 
of a picked-up image in a data area of each image file when 
recording each pickup image on a recording medium. 

[0075] 

Referring to Fig. 2, an image pickup data input 
terminal 201 receives image pickup data obtained by an image 
pickup element having four, R, Gl, G2, and B, color filters 
and converted into digital data. 
[0076] 

This image pickup data is processed in a main image 
processing unit 202 with various image processing operations 
for recording with high image quality as in a digital image 
recording apparatus such as a general digital camera. The 
processed image pickup data is then encoded by an encoding 
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processing unit 203 in accordance with a JPEG recording 

format . 

[0077] 

On the other hand, part of the image pickup data, 
e.g., pixel data of 64 pixels in the central portion of 
the two-dimensional image space of one picture is extracted 
by a color balance data extraction unit 204, as shown in 
Fig. 9. An averaging processing unit 205 generates an 
average value of 16-pixel data of each color component signal 
of R, Gl, G2, or B. 
[0078] 

An image filing unit 206 forms an image file into 
which the resultant average value is converted together 
with encoded image data. The average value is allocated 
to a predetermined color balance data area attached to an 
image file. A medium-recording unit 207 records the average 
value together with the original encoded image data as one 
image file on a recording medium. 
[0079] 

The above operation is always done in all image 
pickup operations including normal image pickup. Color 
balance image pickup is done as one of the general image 
pickup operations. Since color balance data is obtained 
by extracting an image of 64 pixels in the central portion 
of the picture, any color image except color balance data 
may be picked up around the 64-pixel area. 
[0080] 
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A technique for setting in the image pickup apparatus 
a color balance control value used for color balance 
adjustment after white balance adjustment will be described 
below. A medium on which an image file is recorded by 
the above recording scheme is loaded in the image pickup 
apparatus, and image file reproduction is done using a user 
interface shown in Fig. 7. 
[0081] 

A PLAY button 703 shown in Fig. 7 is depressed to 
reproduce and display this image data of the image file, 
on a reproduction monitor 701. 
[0082] 

The operator sequentially switches a plurality of 
files with file selection switches 704 and 705 while viewing 
this reproduction motor 701 and an image 709 superposed 
on the pickup image. The operator then selects one image 
file to be used for color balance adjustment. 
[0083] 

When one image file is reproduced and monitored, 
a color balance data reproduction unit 210 reads out and 
reproduces the color balance data from the attached color 
balance data area. 
[0084] 

File selection is done as follows. While a file 
in which color balance data used for color balance adjustment 
is written is being displayed on the monitor, a SELECT switch 
708 is depressed to cause a color balance data image selection 
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unit 209 to select a desired file. 
[0085] 

As indicated by the monitor 701 in Fig. 7, when 
the image pickup date and place are written in a location 
except a color balance data recording area 702 in 
intentionally picking up an image used for color balance 
adjustment, operation for the color balance data can be 
further facilitated. 
[0086] 

In accordance with an image file selection signal 
indicating that a file is selected, the color balance data 
image selection unit 209 extracts the color balance data 
of an image file being reproduced. A color balance control 
value operating unit 211 calculates a color balance control 
value . 
[0087] 

Color balance calculations are done as follows. 
The ratio of average values of R, Gl, G2, and B is calculated, 
and the reciprocal of the ratio of R and B using the Gl 
and G2 average values as the median is calculated as a color 
balance control value. 
[0088] 

The color balance adjustment control value can be 
finely adjusted (changed) in the following manner. 
[0089] 

For example, adjustment in the R-B direction is 
performed using the switches 704 and 705 shown in Fig. 7. 
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Every time the switch 704 is depressed, a color balance 
adjustment coefficient unit 219 outputs an R sum signal. 

A color balance control value operating unit 211 adds a 
constant to the R control value and subtracts a constant 
from the B control value. Every time the switch 705 is 
depressed, the color balance adjustment coefficient unit 
219 outputs a B sum signal. The color balance control value 
operating unit 211 adds a constant to the B control value 
and subtracts a constant from the R control value. 
[0090] 

Adjustment in the magenta-green direction is 
performed using switches 7 06 and 7 07 shown in Fig . 7 . Every 
time the switch 706 is depressed, the color balance ad j ustment 
coefficient unit 219 outputs an R+B sum signal. The color 
balance control value operating unit 211 adds a constant 
to the R and B control values and subtracts a constant from 
the Gl and G2 control values. Every time the switch 705 
is depressed, the color balance adjustment coefficient unit 
219 outputs a Gr sum signal. The color balance control 
value operating unit 211 adds a constant to the Gl and G2 
control values and subtracts a constant from the R and B 
control values. 
[0091] 

The R, G, and B signals for adjusting the output 
signals of a monitor (not shown) can be similarly controlled 
by the signals from the color balance adjustment coefficient 
unit 219 to allow the operator to confirm the fine adjustment 
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on the monitor. 
[0092] 

The color balance control values obtained as 
described above are set in the image pickup apparatus as 
the color balance adj ustment control values by a color balance 
control value setting unit 212. 
[0093] 

As described above, when image pickup operation 
is performed using the color balance adjustment control 
values thus set, an auto white balance adjustment unit 219 
adjusts the white balance of each color component of a digital 
image signal input from an image pickup data input terminal 
213 via an image pickup element. A color balance adjustment 
unit 214 multiplies the set color balance control value 
with each color component, thereby adjusting the color 
balance . 
[0094] 

For example, when an image has a uniform red portion 
in its center, a reddish image is output since this reddish 
tincture is reflected on the output image. 
[0095] 

The color balance-adjusted signal undergoes color 
processing in a color processing unit 215 and encoding 
processing in an encoding processing unit 216. An image 
filing unit 217 converts the encoded image signal into an 
image file together with the color balance data extracted 
from the image pickup data. A medium-recording unit 218 
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records the image file on a recording medium. 
[0096] 

As described above, there is provided a function 
of setting in the image pickup apparatus the color balance 
adjustment control value obtained from the color balance 
data recorded together with the image pickup data in the 
image file. Color balance data recorded in a plurality 
of scenes can be easily selected and repeatedly used. The 
user can freely express an image by easily changing the 
color tincture of the pickup image. 
[0097] 

As described above, according to the second 
embodiment, color balance data recorded in a plurality of 
scenes can be easily selected and repeatedly used. The 
user can freely change the color balance of the auto white 
balance-adjusted pickup image. The user can freely express 
an image by easily changing the color tincture of the pickup 
image . 
[0098] 

(Third Embodiment ) 

Fig. 3 is a block diagram showing the main part 
of an image pickup apparatus according to the third embodiment . 

This embodiment will explain a method of recording part 
of image pickup data as white sheet data or color balance 
data of a picked-up image in a data area of each image file 
when recording each pickup image on a recording medium, 
and using these data for MWB adjustment and color balance 
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adj ustment . 
[0099] 

Referring to Fig. 3, an image pickup data input 
terminal 301 receives image pickup data obtained by an image 
pickup element (image pickup means) having four, R, Gl, 
G2, and B, color filters and converted into digital data. 
[0100] 

This image pickup data is processed in a main image 
processing unit 302 for various image processing operations 
for recording with high image quality as in a digital image 
recording apparatus such as a general digital camera. The 
processed image pickup data is then encoded by an encoding 
processing unit 303 in accordance with a JPEG recording 
format . 
[0101] 

On the other hand, part of the image pickup data, 
e.g., pixel data of 64 pixels in the central portion of 
the two-dimensional image space of one picture is extracted 
by an image data extraction unit 304, as shown in Fig. 9. 

An averaging processing unit 305 generates an average value 
of 16-pixel data of each color component signal of R, Gl, 
G2, or B. 
[0102] 

An image filing unit 306 forms an image file into 
which the resultant average value is converted together 
with encoded image data. The average value is allocated 
to a predetermined area attached to an image file. A 
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medium-recording unit (recording means) 307 records the 
average value together with the original encoded image data 
as one image file on a recording medium. 
[0103] 

The above operation is always done in all image 
pickup operations including normal image pickup. Each of 
white sheet data image pickup and color balance image pickup 
is done as one of the general image pickup operations. 
Since the white sheet data and color balance data are obtained 
by extracting an image of 64 pixels in the central portion 
of the picture, any image except white sheet data and color 
balance data may be picked up around the 64-pixel area. 
[0104] 

A technique for setting in the image pickup apparatus 
an MWB control value used for MWB adjustment and a color 
balance control value used for color balance adjustment 
will be described below. A medium on which an image file 
is recorded by the above recording scheme is loaded in the 
image pickup apparatus, and image file reproduction is done 
using a user interface shown in Fig. 8. 
[0105] 

A PLAY button 803 shown in Fig. 8 is depressed to 
reproduce and display this image data (image pickup data) 
of the image file on a reproduction monitor 801. 
[0106] 

The operator sequentially switches a plurality of 
files with file selection switches 804 and 805 while viewing 
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this reproduction motor 801 and a white image or color 
adjustment image 810 superposed on the pickup image. The 
operator then selects an image file to be used for white 
balance adj ustment . 
[0107] 

When one image file is reproduced and monitored, 
a selection data reproduction unit 310 reads out and 
reproduces the white sheet data from the attached data area. 
[0108] 

File selection is done as follows. While a file 
in which white balance data used for white balance adjustment 
is written is being displayed on the monitor, a WHITE switch 
808 is depressed to cause a CB/WB adjustment image selection 
unit (first selection means) 309 to select a desired file. 
[0109] 

As indicated by the monitor 801 shown in Fig. 8, 
if the image pickup date and place are written in a location 
except a white balance data recording area 802 when 
intentionally picking up an image used for white balance 
adjustment, operation for the white sheet data can be further 
facilitated. This also applies to color balance adjustment 
to be described later. 
[0110] 

The attached data of the image file being reproduced 
is selected by the CB/WB adjustment image selection unit 
309 in accordance with the file selection signal indicating 
that a file is selected. Upon depressing the White switch 
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808, a switch 321 is switched in accordance with a signal 
from a CB/WB selection unit 320 for switching between white 
balance data and the color balance data. A white balance 
control value operating unit 311 calculates a white balance 
control value using the selected data. 
[0111] 

White balance calculations are done as follows. 
The ratio of average values of R, Gl, G2 , and B is calculated, 
and the reciprocal of the ratio of R and B using the Gl 
and G2 average values as the median is calculated as a white 
balance control value. 
[0112] 

The white balance adjustment control value can be 
set in the image pickup apparatus as the MWB adjustment 
control value by a CB/WB control value setting unit 313. 
[0113] 

When the MWB control value is set in the image pickup 
apparatus, a color balance control value is calculated in 
the same manner as in the MWB control value and set in the 
image pickup apparatus. 
[0114] 

The PLAY button 803 shown in Fig. 8 is depressed 
to cause a medium-reproduction unit 308 to reproduce and 
display image pickup data on a reproduction monitor 701. 
[0115] 

The operator sequentially switches a plurality of 
files with file selection switches 804 and 805 while viewing 
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the pickup image displayed on the reproduction motor 801. 

The operator then selects one image file desired to be 
used for color balance adjustment. 
[0116] 

While the image file is reproduced and monitored, 
a selection data reproduction unit 310 reads out the color 
balance data from the attached data area and reproduces 
it on the monitor 701. 
[0117] 

File selection is done as follows. While a file 
in which data desired to be used for color balance adjustment 
is being displayed on the monitor, a COLOR switch 809 is 
depressed to cause the CB/WB adjustment image selection 
unit 309 to select a desired file. 
[0118] 

In accordance with the file selection signal 
indicating that a file is selected, the CB/WB adjustment 
image selection unit 309 selects the attached data of the 
image file being reproduced. Upon depressing the COLOR 
switch 809, the switch 321 is switched in accordance with 
the WB/CB selection unit 320 for switching between the white 
balance data and the color balance data. A color balance 
control value operating unit 312 calculates a color balance 
control value using the selected data. 
[0119] 

Color balance calculations are done as follows. 
The ratio of average values of R, Gl, G2, and B is calculated, 
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and the reciprocal of the ratio of R and B using the Gl 
and G2 average values as the median is calculated as a color 
balance control value. 
[0120] 

The color balance adjustment control value can be 
finely adjusted in accordance with a signal from a color 
balance adjustment coefficient unit 322 as in the second 
embodiment . 
[0121] 

The color balance control values thus obtained are 
set in the image pickup apparatus by the CB/WB control value 
setting unit 313 as the color balance adjustment control 
values . 
[0122] 

When image pickup operation is performed using the 
white balance adjustment control value and color balance 
adjustment control value set as described above, a white 
balance adjustment unit 320 performs MWB adjustment for 
each color component of a digital image signal input from 
an image pickup data input terminal 314 via an image pickup 
element, thereby adjusting the white balance. 
[0123] 

A color balance adjustment unit 315 then performs 
color balance adjustment. The set color balance control 
value is multiplied with each color component to perform 
color balance adjustment. 
[0124] 
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The color balance-adjusted signal undergoes color 
processing in a color processing unit 316 and encoding 
processing in an encoding processing unit 317. An image 
filing unit 318 converts the encoded image signal into an 
image file together with the white sheet data and color 
balance data extracted from the image pickup data. A 
medium-recording unit 319 records the image file on a 
recording medium. 
[0125] 

As described above, there is provided a function 
of setting in the image pickup apparatus the MWB adjustment 
control value and color balance adjustment control value 
obtained from the color balance data recorded together with 
the image pickup data in the image file. Attached data 
of image files recorded in a plurality of scenes can be 
easily selected and repeatedly used. The user can freely 
express an image by easily changing the color tincture of 
the pickup image. 
[0126] 

More specif ically, MWB white sheet data is read 
out to perform white balance adjustment. The same white 
sheet data or another white sheet data also can be read 
out to perform color balance adjustment. Thus, the color 
tone of the pickup image can be freely controlled. 
[0127] 

Thus, according to the third embodiment, MWB white 
sheet data is read out to perform white balance adjustment. 
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The same white sheet data or another white sheet data also 
can be read out to perform color balance adjustment. Thus, 
attached data of image files recorded in a plurality of 
scenes can be easily selected and repeatedly used. The 
user can freely express an image by easily changing the 
color tincture of the pickup image. 
[0128] 

( Fourth Embodiment ) 

Fig. 4 is a block diagram showing the main part 
of an image pickup apparatus according to the fourth 
embodiment. This embodiment will explain a method of 
recording part of picked-up image as gain control data for 
MWB and color balance adjustment in an attached data area 
of each image file, performing adjustment operation so as 
to use the gain adjustment data in any image pickup apparatus 
and perform MWB adjustment and color balance adjustment 
in recording each pickup image on a recording medium. 

[0129] 

A method of extracting some data from image pickup 
data and recording the extracted data as data for MWB or 
color balance adjustment in an attached area of each image 
file is the same as in the first embodiment, and a detailed 
description thereof will be omitted. 
[0130] 

According to the characteristic feature of the fourth 
embodiment, image pickup element fluctuation adjustment 
data recorded in an image pickup element fluctuation 
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adjustment data recording area (memory) or adjustment data 
unit 419 of this image file as data obtained by picking 
up images of a white sheet under each of two different specific 
light sources are recorded in a specific part of the attached 
area . 
[0131] 

Image pickup element fluctuation adjustment data, 
i.e., image pickup data 1203 and 1205 of two different light 
sources obtained by using a reference image pickup apparatus 
shown in Fig. 10 and image pickup data 1204 and 1206 of 
two different light sources unique to each image pickup 
apparatus for each color component of R, Gl, G2, or B, are 
recorded by the adjustment data unit (memory) 419 in the 
area attached to the image file. This embodiment uses two 
different light sources. However, adjustment precision 
is improved when the number of types of light sources increases . 

Although the adjustment precision is not improved so much, 
the image pickup element fluctuation adjustment can be also 
attained by using only one kind of light source. 

[0132] 

These image pickup element fluctuation adjustment 
data which is written in the memory of each image pickup 
apparatus at the time of shipment, are read out by a readout 
means (not shown) ( second readout means ) in each image pickup 
operation and then written in a single file together with 
the MWB white sheet data and color balance data. 
[0133] 
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The average value of all or some image pickup data 
for each color component of R, Gl, G2 , or B, obtained when 
picking up the image of a light source, is used as such 
image pickup element fluctuation adjustment operation data. 
[0134] 

As in the above three embodiments, when the attached 
data of a file selected by a selection data reproduction 
unit 410 shown in Fig. 4 is read out, the fluctuation 
adjustment data are also read out and reproduced. 
[0135] 

An adjustment operation unit 420 adjusts these 
fluctuation adjustment data in the following manner and 
supplies the resultant data as R, Gl, G2, and B data for 
MWB or color balance adjustment to a gain control value 
operating unit 4111. The gain control value operating unit 
4111 calculates gain control values for MWB and color balance 
ad j ustment . 
[0136] 

Referring to Fig. 10, an image pickup apparatus 
1201 records MWB and color balance adjustment data on a 
medium. An image pickup apparatus 1202 reads out the MWB 
and color balance gain adjustment data recorded on the 
recording medium and performs gain adjustment. 
[0137] 

Fluctuation adjustment between the image pickup 
elements in an image pickup apparatus is two-dimensionally 
performed using ratios R/B and (R+B) / (G1+G2 ) of reference 
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data Rrefa, Glrefa, G2refa, Brefa of a light source a, 
reference data Rrefb, Glrefb, G2refb, and Brefb of a light 
source b, and data Ra, Gla, G2a, and Ba of the light source 
a, and data RbS, GlbS, G2bS and BbS of the light source 
b which are unique to the image pickup apparatus. 
[0138] 

The method of fluctuation adjustment calculation 
between the image pickup elements in an image pickup apparatus 
can be expressed by a difference such as an (R-B) value 
or (R+B) - (G1+G2) value. Any mathematical expression can 
be used if it can be expressed so as to absorb fluctuations 
between the image pickup elements in the image pickup 
apparatus . 
[0139] 

First of all, the relationship between the reference 
data RrefaS, GlrefaS, G2refaS, BrefaS, RrefbS, GlrefbS, 
G2refbS, and BrefbS (1203) read out from a given recording 
medium and adjustment data RaS, GlaS, G2aS, BaS, RbS, GlbS, 
G2bS, and BbS (1204) unique to the image pickup apparatus 
and recorded on the given recording medium is adjusted using 
MWB and color balance gain adjustment data R, Gl, G2, and 
B read out from the given recording medium. 
[0140] 

Gain values and offset values of the difference 
between the R/B ratios and (R+B) / (G1+G2) ratios of the light 
sources a and b are obtained: 
[0141] 
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GainRB = { (Rref aS/Bref aS) 

- (RrefbS/BrefbS) }/{ (RaS/BaS) 

- (RbS/BbS) } ... (1) 
OffsetRB = (RrefaS/BrefaS) - GainRB* (RaS/BaS) 
... (2) 

These are reflected on the gain adjustment data. 
[0142] 

( R 1 /B 1 ) - GainRB* (R/B) + OffsetRB ---(3) 
Similarly, the following relations are obtained for the 
(R+B) / (G1+G2) ratios : 

GainRBGlG2 = { (Rref aS+Bref aS) / (Glref aS+G2ref aS) 

- (RrefbS+BrefbS) / (Glref bS+G2refbS ) }/ 
{ (RaS+BaS) / (GlaS+G2aS) - (RbS+BbS) / 

(GlbS+G2bS) } ... (4) 

Off setRBGlG2 - (Rref aS+Bref aS ) / (Glref aS+G2refaS ) } 
- GainRBGlG2* { (RaS+BaS) / (GlaS+G2aS) } 

... (5) 

(R» +B' ) / (Gl ' +G2 ' ) = GainRBGlG2* { (R+B) / (G1+G2) } 

+ OffsetRBGlG2 ... (6) 

These equations yield data converted into reference 
data for the image pickup apparatus which recorded the data 
on the medium. This is the adjustment operation 1207. 
[0143] 

The adj ustment operation (2) 12 08 and the ad j ustment 
operation (3) 1209 are obtained from the equations for 
obtaining the completely same gain values and offset values 
in the adjustment operation (1) 1207. 
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[0144] 

Now it is assumed that (R ,, /B ,, ) / 
(R f 1 +B' 1 ) / (Gl 1 ' +G2' 1 ) can be calculated from the adjustment 
operation (2) for calculating (R'/B'), (R 1 +B 1 ) / (Gl 1 +G2 ' ) , 
and (R ,r, /B f,! )/ (R 1 1 1 +B 1 1 1 ) / (Gl 1 1 1 +G2 ' 1 f ) can be 
calculated from the adj ustment operation (3) for calculating 
( R 1 1 /B 1 ' ) , (R f ' +B' ' ) / (Gl 1 '+G2' ' ) . 
[0145] 

Since Gl 1 1 1 and G2 1 1 1 are the center of the RG1G2 
ratio, they are made to equal to the GlaD and G2aD values 
of an image pickup apparatus 202 (medium-use-side image 
pickup apparatus D) . Thus, R f 1 1 and B 1 1 1 values can be 
calculated . 
[0146] 

The R T ' 1 r Gl ' ' ' , G2 f 1 ' , and B 1 1 f values thus obtained 
represent values which absorb fluctuations between the image 
pickup elements in the image pickup apparatuses 1201 and 
1202 upon converting the data of the image pickup apparatus 
1201 (medium-providing-side image pickup apparatus S) into 
data of the image pickup apparatus 1202 (medium-use-side 
image pickup apparatus D) . 
[0147] 

The gain adjustment control value operation unit 
411 calculates the ratio of R 1 1 1 , Gl 1 1 1 , G2 ' ' 1 , and B 1 1 ' 
from the values obtained by the arithmetic operations (1) , 
(2) and (3) and obtains the MWB and color balance adjustment 
control values. Then, the gain control value setting unit 
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(setting means) 412 performs settings. 
[0148] 

Using the gain adjustment control values thus set, 
the gain adjustment unit 414 performs MWB or color balance 
adjustment in the subsequent image pickup. The data are 
converted into an image file as in the above embodiments. 

The resultant data is recorded on a recording medium by 
a medium-recording unit 418- 
[0149] 

By the above method, data written in the specific 
area attached to the image file used for MWB and color balance 
adjustment is adjusted for fluctuations using the image 
pickup element fluctuation adjustment operation data of 
the image pickup apparatus, written in the same specific 
area. Thus, as long as identical light sources are used, 
thesameR, G, Gl, and B values are obtained even when different 
image pickup apparatus are used. Therefore, data 
fluctuations between the image pickup apparatuses can be 
absorbed . 
[0150] 

As described above, even if MWB or color balance 
control values are obtained using the white sheet data or 
color balance adjustment data of image files picked up using 
different image pickup apparatuses, optimal (accurate) MWB 
or color balance adjustment operation can be done. 
[0151] 

According to the fourth embodiment, as described 
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above, white balance adjustment or color balance adjustment 
using white sheet data can be accurately done in an image 
pickup apparatus different from that by which the white 
sheet data are obtained. 
[0152] 

In the first to fourth embodiments, the colors of 
the color filters of the image pickup element are R, Gl, 
G2, andB. However, the present invention is also applicable 
to complementary color filters of Mg (magenta) , Gr (green) , 
Cy (cyan) , and Ye (yellow) components, as shown in Fig. 6. 
[0153] 

The object of the present invention is realized 
even by supplying a storage medium storing software program 
codes for realizing the functions of the above-described 
embodiments to a system or an apparatus, and causing the 
computer (or a CPU or an MPU) of the system or the apparatus 
to read out and execute the program codes stored in the 
storage medium. 

In this case, the program codes read out from the 
storage medium realize the functions of the above-described 
embodiments by themselves, and the storage medium storing 
the program codes therefore constitutes the present 
invention . 
[0154] 

As a storage medium for supplying the program codes, 
a floppy disk, a hard disk, an optical disk, a magnetooptical 
disk, a CD-ROM, a CD-R, a magnet ic tape, a nonvolatile memory 
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card, a ROM, or the like can be used. 

The functions of the above-described embodiments 
are realized not only when the readout program codes are 
executed by the computer but also when the OS (Operating 
System) running on the computer performs part or all of 
actual processing on the basis of the instructions of the 
program codes. 
[0155] 

The functions of the above-described embodiments 
are also realized when the program codes read out from the 
storage medium are written in the memory of a function 
expansion board inserted into the computer or a function 
expansion unit connected to the computer, and the CPU of 
the function expansion board or function expansion unit 
performs part or all of actual processing on the basis of 
the instructions of the program codes. 
[0156] 

When the present invention is applied to the above 
storage medium, program codes corresponding to the flow 
chart described above are stored in the storage medium. 

Briefly speaking, modules indispensable to the image pickup 
apparatus of each embodiment described above are stored 
in the storage medium. 
[0157] 

[Effect of the Invention] 

As has been described above, according to the present 
invention, convenience of the white sheet data used for 
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color correction of a picked-up image can be improved. 
[Brief Description of the Drawings] 
[Figure 1] 

A block diagram showing the main part of an image 
pickup apparatus according to the first embodiment. 
[Figure 2] 

A block diagram showing the main part of an image 
pickup apparatus according to the second embodiment. 
[Figure 3] 

A block diagram showing the main part of an image 
pickup apparatus according to the third embodiment. 
[Figure 4] 

A block diagram showing the main part of an image 
pickup apparatus according to the fourth embodiment. 
[Figure 5] 

A view showing the structure of primary color filter 
arrays of image pickup elements. 
[Figure 6] 

A view showing the structure of complementary color 
filter arrays of image pickup elements. 
[Figure 7] 

A view showing the arrangement of a user interface 
according to the first and second embodiments. 
[Figure 8] 

A view showing the arrangement of a user interface 
according to the third embodiment. 
[Figure 9] 
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A view showing image pickup data to be extracted 
according to the first to fourth embodiments. 
[Figure 10] 

A view for explaining the order of operation for 
adjusting variations in image pickup elements according 
to the fourth embodiment. 

[Figure 11] 

A block diagram showing a prior art. 
[Figure 12] 

A block diagram showing another prior art. 
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[Name of the Document] Abstract 

[Abstract] 

[Object] 

An object of the present invention is to provide 
an image pickup apparatus in which convenience of the white 
sheet data used for color correction of a picked-up image 
can be improved. 

[Means for Achieving the Object] 

Each input image pickup data is converted into a 
file by an image filing unit 106 and recorded by a 
medium-recording unit 107 . Average value data of each color 
component data for part of this picked-up data is stored 
in an area different from the image pickup data recording 
area in each file . When desired image pickup data is selected 
by a white sheet data image selection unit 109 by the use 
of a monitor for reproducing and displaying the image pickup 
data, the average value data stored in the same file as 
the image pickup data is read out by a white sheet data 
reproduction unit 110. A control value of a white balance 
adjustment unit 114 is calculated by a white balance control 
value operating unit 111 on this basis of the data. This 
control value is set by a white balance control value setting 
unit 112, and white balance adjustment is performed by the 
use of the set control value. 
[Elected Drawing] 

Figure 1 



